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FOREWORD 



The ENLISTED MANPOWER Work Unit conducts a continuing research program to 
maintain and improve techniques and procedures for screening potential enlisted man- 
power. Objectives are 1) to develop new forms of screening measures for use by the 
Army and the other services so as to assess more effectively the trainability and usa- 
bility of potential enlisted personnel; 2) to develop new reference measures for use as 
standards in developing screening and classification tests for all the services; and 
3) to improve methods for extracting predictive information from screening tests. 

As one avenue to development of technical information that can contribute to more 
effective input screening, the feasibility of programmed testing is being investigated. 
8ESRL has conducted several experimental and theoretical studies of branching tests in 
which testing is individualized by having test questions so programmed that en examinee 
who answers a test item correctly is presented next with a more difficult item and an 
examinee who answers incorrectly is presented with an easier item. By contrast, in 
conventional tests all examinees answer the same items presented in the same order, 
The present publication reports on a comparison of a variety of computer-simulated 
conventional and branching tests, 

The entire ENLISTED MANPOWER Work Unit is responsive to special requirements of 
the Deputy Chief of Staff for Personnel, as well as to requirements of RDT&E Project 
2Q02<*701A721, "Selection and Behavioral Evaluation," fY 1970 Work Program. 
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COMPARISON OF COMPUTER-SIMULATED CONVENTIONAL AND BRANCHING TESTS 



BRIEF 



Requirement: 

To compare a variety of computer-simulated conventional and branching tesis and 
to extend the theoretical analysis of branching techniques. 



Procedure: 

Computer-simulated tests of two typos were compared. One type consisted of con- 
ventional testfi varying in length (5, 10, 15 items) and distribution of item difficulty 
indexes (ell items at p s .50; normal, p * .30 - .70 and p * .10 - .90; and rectilinear, 
p s .30 - .70 and p * .10 - -90) . The second consisted of branching tests varying in length 
{5, 10, 15 items to be answered by each examinee) , number of items presented at each 
level of difficulty (1.2), and distribution of item difficulties comparable to those of the 
conventional tests. In addition, both types of test were varied in assumed item validity. 
The comparisons were made in terms of correlation between test scores and underlying 
ability (L0it'’s model) . 



Findings. 

In tests with higher item validities (r^j s « .60 - .90) , a branching test had a higher 
correlation with underlying ability than did any conventional test, for all three lengths 
studied. 



Applicability: 

This theoretical analysis supports an exploratory experimental study previously 
made. It indicates the research promise of tests with branching programs and provides 
useful guidelines for the design of further studies of programmed tests. 
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COMPARISON OF COMPUTER-SIMULATED CONVENTIONAL AND BRANCHING TESTS 



In line with its interest in unconventional testing techniques, the 
Behavior and Systems Research Laboratory (BRSRL) has conducted 
several research studies of the branching techniques. Waters ( 1 ), in a 
theoretical study, found that a hypothetical five-item b anching test 
correlated somewhat higher with underlying ability than lid hypothetical 
five-item conventional tests. Bayroff and Seeley ( 2 ) obtained empirical 
indications that correlation between short branching tests and long con- 
ventional tests was considerably higher than the expected correlation 
between equally short conventional tests and long conventional tests. 

The present report continues the theoretical analysis aid compares a 
variety of computer- simulated conventional and branching tests. 

V 

In the usual testing situation, each examinee takes all the items, 
and item sequence is the same for each examinee. It if possible, how- 
ever, to have sequential or branching tests in which a r ' 1 examinees do 
not take the same items and the sequence of item presentation for an 
individual is some function of his performance on previous items; that 
is, an item answered correctly is followed by a more difficult: item, an 
item answer incorrectly, by a less difficult item. Tie rationale for 
the latter procedure is that presentation of items bai ed on an examinee r s 
past performance allows each individual to take items that are progres- 
sively more appropriate to his own level of ability. It is conceivable 
that such a procedure v?ould reduce testing time, and for a given amount 
of time would permit more accurate measurement of an individual 1 s ability, 
principally by reducing opportunities for chance success by low ability 
examinees' attempting items too difficult for them. 



TESTS 

Conventional Tests 

Five-, ten-, and fifteen-item hypothetical con/entional ( C) tests 
were evaluated. All tests were symmetric around p * . 50 , but varied in 
item difficulty distributions. The distributions investigated were all 
items at p = .50 (C50), roughly normal (CN), or rectilinear ( CR) . Each 
of the CN and CR tests was tried out with difficulty ranges of .30 
through .70 and .10 through .90. Table 1 gives the C5O, CN, and CR 
Item difficulty distributions for the five-, ten-, and fifteen-item 
conventional tests. \ 



NUMBER OF ITEMS AT EACH P- VALUE FOR CONVENTIONAL TESTS 



0 
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No five-item normally distributed tests were evaluated. 



Branching Tests 



One- I tern- Per -Stage Tests , Six hypothetical one- i tem-per-s tage (l-PS) 
branching tests were evaluated. The structure of these six tests is shown 
in Figures 1 through 6, together with the difficulty of each item. Two 
tests, were studied at each of the three tests lengths ( 5, 10, and 15 items) . 
One of the two tests covered a difficulty range of .30 through ,70 and 
the other ranged from .10 through .90 . The ::iva-item-per-sub jec t branch- 
ing tests contained I5 items with each examiiee responding tc only five 
of the items. In the ten- item- per- subject tasts, each examinee took ten 
of the 55 items in the test. The fifteen-item-per-subject tests were 
composed of 120 items. In each of the six tests, the first item (p == .50) 
was the same for all examinees, but the remaining items taken were deter- 
mined by the examinee’s performance on the immediately preceding item. 

If en examinee passed an item, he proceeded to a more difficult one; if 
he failed an item, he proceeded to an easiei one. When the range of 
p-vf,lues in a test was .JO through .70, increases and decreases in diffi- 
culty between adjacent items were in steps of .05 for the f ive-i tem-per- 
subject test, .0222 for the ten- item- per- subject test, and .0143 for the 
f if >:een- i tem-per-sub jec t test. For the .10 through .90 range tests, the 
stews were .10 for the f ive- i tem-per-subj ec 1 ; test, .0444 for the ten-item- 
per -subject test, and .0286 for the f if teen -i tem-per-sub ject test. 

Two-Item- P er-Stage Tests . Four hypothetical two- item-per- stage 
( 2 -PS), ten-i tem-per- sub jec t branching tests were evaluated. The st..uc- 
tur* of these four tests, and the distributions of item difficulties, is 
shown in Figure 7 * Each of these tests was composed of 114 items. At 
eaci stage in these tests, the examinees took two items of the same dif- 
ficulty level . The first two items taken by all examinees had p-values 
of .50. If the examinee passed both items in a pair, he branched to a 
more difficult item pair; if he passed one of the items in a pair, he 
branched to a pair of equal difficulty; if he failed both items in a 
pair, he proceeded to an easier pair of items. Items for two of 
the tests covered a difficulty range of .3C through .70, while the other 
twc tests ranged from .10 through .90. Foi each of these difficulty 
ranges, one branching tests was developed t-y having equally spaced item 
pairs in the terminal row of the test ( 2 -PS-E). The p-values of the item 
pa .rs in the other rows were determined from the terminal item pair values. 
Fo: the other two- item- per-stage tests, 2 - 7 S-U (one for each of the item 
difficulty ranges), the item pair p-values were determined by branching 
downward from the p = .50 item pair to the terminal row of item pairs. 

Using this procedure, the item pairs in the terminal rows were not 
equally spaced as in the 2 -PS-E tests but were spaced so that the in- 
tervals between item pairs were smaller ir the middle part of the diffi- 
cilty ranges, and larger nearer the extreme difficulty values. Scores 
for all four of the two- item- per-stage tents ranged from 0 to 62 . 
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SCORING SCALC 



Figure 1. One-item-per-$tage, five-iiam branching test with a difficulty range* 
of .30*. 70 



DIFFICULTY VALUES FOR 


items sho;n 


IN FIGURE 1 


Item No. 


Diff. Value 


Item No. 


Diff Value 


i 


• 5000 


9 


.4500 


2 


.5500 


10 


•5500 


5 


• 4500 


11 


.7000 


4 


.6000 


12 


.6000 


5 


.5000 


15 


.5000 


6 


.4000 


14 


.4000 


7 


.65CO 


15 


■ 5000 


8 


• 5500 
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SCORING SCALE 

Figure 2. One-item-per-stage, five-item branching test with a difficulty range 
of .1 0-.90 



DIFFICULTY VALUES FOR 


ITEMS SHOWN 


IN FIGURE 2 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


1 


• 5000 


9 


.4000 


2 


.6000 


10 


.2000 


z 

s 


.4000 


11 


.9000 


4 


.7000 


12 


.7000 


r 

> 


.5000 


13 


.5000 


6 


.JOOO 


14 


.5000 


7 


. 80 <\) 


15 


.1000 


3 


.6000 
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SCORING SCALE 



Figure 3. One-itenvper-stage, ten-item branching test with a difficulty range of .30-. 70 
(Qifficufty values for items shown are given nn page 7* 
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DIFFICULTY VALUES FOR ONE- ITEM- PER STAGE, TEN-ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE uF .30-. 70 (Figure 3) 

Item No. DIff. Value Iterr. No. Diff. Value Item No. Diff. Value 


1 


• 5000 


19 


.4778 


37 


•6776 


2 


.5222 


20 


.4334 


38 


• 6330 


3 


.4778 


21 


.3890 


39 


.5888 


4 


.5444 


22 


■6330 


40 


.5444 


5 


.5000 


23 


.5888 


41 


.5000 


6 


. 45^,6 


24 


.5144 


42 


.4556 


7 


.5666 


25 


• 5000 


43 


.4112 


8 


.5222 


26 


•4556 


44 


.3658 


9 


,4778 


27 


.4112 


45 


.3224 


10 


.4334 


28 


• 3668 


46 


.6998 


11 


.3888 


29 


.6554 


47 


■6554 


12 


.5444 


30 


.6110 


48 


.6110 


13 


.5000 


31 


. 3666 


49 


.5666 


14 


.4556 


32 


.5222 


50 


.5222 


15 


.4112 


33 


.4778 


51 


.4778 


16 


.6110 


34 


■4334 


52 


■ 4334 


17 


.3666 


35 


.3890 


53 


.3890 


18 


.5222 


36 


.3446 


54 


.3446 










55 


.3002 
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SCORING SCALE 

Figure 4. One-itenvper-stage, ten-item branching test vuth a difficulty range of .10-.9Q 
(Difficulty values for items shown are given on pnge 91 
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DIFFICULTY VALUES FOR ONE- ITEM- PER- STAGE , TEN- ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE 07 .10- -90 (Figure 4) 



Item No. 


Diff. Value 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


1 


.5000 


19 


.4556 


57 


.8552 


2 


.5444 


20 


.5668 


58 


.7664 


5 


.4550 


21 


.2780 


59 


.6776 


4 


.5888 


22 


.7664 


40 


.5888 


5 


.5000 


25 


.6776 


41 


.5000 


6 


.4112 


24 


.58.38 


42 


.4112 


7 


.6552 


25 


.5000 


45 


.5224 


8 


.5444 


26 


.4112 


44 


.2556 


9 


.4556 


27 


.5224 


45 


.1448 


10 


.5668 


28 


• 2556 


46 


.8996 


11 


• 6776 


29 


.8108 


47 


.8108 


12 


.5888 


50 


.7220 


48 


.7220 


15 


.5000 


51 


•6552 


49 


.6552 


14 


.4112 


52 


.5444 


50 


.5444 


15 


.5224 


55 


•4556 


51 


.4556 


16 


.7220 


54 


.5668 


52 


.5668 


17 


• 6552 


55 


.2780 


55 


.2780 


18 


.5444 


56 


.1892 


54 


.1892 



55 .1004 
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IS 14 15 12 11 

A A AAA 

21 20 19 18 IT 16 

/\/\/\/\/\/\ 

28 27 26 25 24 23 22 

/\/\/\ /\/\/\/\ 

3G 35 34 33 32 31 30 29 

AAA A A AAA 

45 44 43 42 41 40 59 38 37 

AAA A A AAA A 

55 54 53 52 51 50 49 48 47 46 

A A A A A A A A A A 

66 65 64 63 62 61 60 59 58 57 56 

A A A AAA A A A A A 

; 8 77 76 75 74 73 72 71 70 69 68 67 

\AA/\ A A A A A A A A 

91 qo 89 83 &7 86 85 84 83 82 81 SO 79 

A A A A A A A A A A AAA 

105 104 103 102 101 100 99 98 97 96 95 94 93 92 

A A A A A A A A A A A A/\ A 

120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 

A A A A A A A A A A A A A A A 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 

SCORING SCALE 

Figure 5. One-item-per-stage, fifieernlem branching test with a difficulty range of .30-. 70 
( Difficulty values for items shown are given on pages 11 and 12) 
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DIFFICULTY VALUES FOR ONE- ITEM- FER-STAGE, FIFTEEN-ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .30- -70 (Figure 5) 



:em No. 


Dlff. Value 


1 


.5000 


2 


.5143 


3 


.4857 


4 


.5286 


5 


.5000 


6 


.4714 


7 


• 5429 


8 


• 5143 


9 


.4857 


10 


.4571 


11 


• 5572 


12 


.5286 


13 


.5000 


14 


.4714 


15 . 


.4428 


16 


.5715 


17 


.5429 


18 


.5143 


19 


.4857 


20 


.4571 



:em No . 


Diff. Value 


21 


.4285 


22 


.5858 


23 


•5572 


24 


.5286 


25 


.5000 


26 


.4714 


27 


.4428 


28 


.4142 


29 


.6001 


30 


•5715 


31 


.5429 


32 


.5143 


33 


.4857 


34 


.4571 


35 


.4285 


36 


• *599 


37 


.6144 


38 


.5858 


39 


•5572 


40 


.5286 



Item No . Dlff* Value 



41 


.5000 


42 


.4714 


43 


.4428 


44 


.4142 


45 


.3856 


46 


.6287 


47 


.6001 


48 


.5715 


49 


■5429 


50 


• 5143 


51 


.4857 


52 


.4571 


53 


.4205 


54 


•3999 


55 


.3713 


56 


.6430 


57 


.6144 


58 


.5858 


59 


•5572 


60 


.5286 
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DIFFICULTY VALUES FOR ONE- ITEM- PER- STAGE, FIFTEEN- ITEh BRANCHING TEST 
WITH DIFFICULTY RANGE OF .50-.70 (Continued) 



Item No. 


Diff. Value 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


61 


.5000 


81 


.6144 


101 


.4285 


62 


.4714 


82 


.5858 


102 


.3999 


63 


.4428 


83 


.5572 


103 


.3713 


64 


.4142 


84 


.5286 


104 


.3427 


65 


.3856 


85 


.5000 


105 


.3141 


66 


•3570 


86 


.4714 


106 


.7002 


67 


•6573 


87 


.4428 


107 


. 671 6 


68 


.6287 


88 


.4142 


108 


.6430 


69 


.6001 


89 


• 3856 


109 


.6144 


70 


•5715 


90 


•3570 


110 


.5858 


71 


•5429 


91 


.3284 


111 


.5572 


72 


• 5143 


92 


•6859 


112 


.5286 


73 


.4857 


93 


• 6573 


113 


.5000 


74 


.4571 


94 


.6287 


114 


.4714 


75 


.4285 


95 


.6001 


115 


.4428 


76 


•3999 


96 


.5715 


116 


.4142 


77 


.3713 


97 


.5429 


117 


.3856 


78 


•3427 


98 


.5143 


118 


•3570 


79 


.6716 


99 


.4857 


119 


.3284 


80 


.6450 


100 


.4571 


120 


.2998 
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1*5 14 13 12 11 

A/\/\/\/\ 



21 20 19 18 17 13 

/\/\/\/\/\/\ 

28 27 26 25 24 23 i 22 

/\/\/\/\/\/v/\ 

36 35 M 33 32 31 '0 29 

/\/\/\/\/\/\/\/\ 

A 44 43 42 41 40 39 38 37 

/\/\ AAAAAA 

55 54 53 c i2 51 50 49 18 47 46 

AAA A A A A A A A 

66 65 64 63 62 61 60 3? 5^ 57 56 

A A AAA A A A AAA 

78 77 76 75 74 73 72 yl 7° 69 68 67 

AAAAAA A A A AA/\ 

91 90 89 88 87 86 85 84 83 82 61 80 7? 

/\/\A A/\ AA A A A AA/\ 

105 104 103 102 loi 100 00 98 97 96 95 94 93 92 

AAA A AAA A A A A AAA 

120 119 118 117 116 115 114 115 112 111 110 1C9 108 107 106 

AAAAAA A A A AAA AAA 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 

SCORING SCALE \ 

Figure 6. One-item-por-stage, f ifteen-it am branching test with a difficult * range of ,10^.90 
f difficulty vafues for items shown ate given on pages 14 and 1 51 
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DIFFICULTY VALUES FOR ONE- ITEM- PER- STAGE, FIFTEEN- ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .10-. 90 (Figure 6) 



Item No. 


Diff. Value 


Item No. 


D.f f . Value 


Item No. 


Diff. Value 


1 


.5000 


21 


.3572 


41 


.5000 


2 


• 5285 


22 


.6713 


42 


.4429 


3 


.4714 


23 


.6142 


43 


.3858 


4 


• 5371 


24 


•5571 


44 


.3287 


5 


. 5000 


25 


.5000 


45 


.2716 


e 


.4429 


26 


.4429 


46 


.7569 


7 


.5856 


27 


.3858 


47 


.6998 


8 


.5285 


28 


.3287 


48 


.6427 


9 


.4714 


29 


.6998 


49 


.5856 


10 


.4143 


30 


.6427 


50 


.5285 


11 


.61 12 


31 


.5856 


51 


.4714 


12 


.5571 


32 


.5285 


52 


.4143 


13 


• 5000 


33 


.4714 


53 


. 3572 


14 


.4429 


34 


.4143 


54 


.3001 


15 


.3858 


35 


•3572 


55 


.2430 


16 


.6427 


36 


.3001 


56 


• 7855 


17 


.5856 


37 


.7284 


57 


.7284 


18 


.5285 


38 


.6713 


58 


. 6713 


19 


.4714 


39 


.6142 


59 


.6142 


20 


.4143 


40 


.5571 


60 


• 5571 
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DIFFICULTY VALUES FOR ONE- ITEM- PER- ST AGE , FIFTEEN-ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .10- .90 (Continued) 



Item No, 


Diff. Value 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


61 


■ 5000 


81 


.7284 


101 


.3572 


62 


.4429 


82 


.6713 


102 


.3001 


63 


.3858 


83 


.6142 


103 


.2130 


64 


.3287 


84 


•5571 


104 


.I859 


65 


.2716 


85 


• 5000 


105 


.1288 


66 


.2145 


86 


.4429 


106 


.8997 


67 


.8140 


87 


.3858 


107 


.8426 


63 


•7569 


88 


.3287 


108 


.7855 


69 


.6998 


89 


.2716 


109 


.7284 


70 


.6427 


90 


.2145 


110 


•6713 


71 


•5856 


91 


.1574 


111 


.6142 


72 


.5285 


92 


.87.ll 


112 


•5571 


73 


.4714 


93 


.8140 


113 


.5000 


74 


.4143 


94 


•7569 


114 


.4429 


75 


•3572 


95 


. 6998 


115 


.3858 


76 


• 3001 


96 


.6427 


116 


.3287 


77 


.2430 


97 


. 5856 


117 


.2716 


78 


•1859 


98 


.5285 


118 


.2145 


79 


.8426 


99 


.4714 


119 


•1574 


80 


•7855 


100 


.4143 


120 


.1003 
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Figure 7. Diagram of two- item-per- stage, ten-item branching tests of varying difficulty distributions 

(Dtstn buttons of item difficulties are given on pages 17 thru 24) 



DIFFICULTY VALUES FOR TOO-ITEM- 
WITH DIFFICULTY RANGE 

Item No. Diff. Value Item No. 


PER-STAGE-E, TEN-ITEM BRANCHING TEST 
OF .30-. 70 (Figure 7) 

Diff. Value Item No. Diff. Value 


1 


• 5000 


20 


.3933 


39 


• 4733 


2 


.5000 


21 


.3400 


40 


• 4733 


5 


.6067 


22 


.3400 


41 


.4467 


4 


.6067 


23 


.6867 


42 


.4467 


5 


.3000 


24 


.6867 


43 


.4200 


6 


.5000 


25 


.6600 


44 


.4200 


7 


• 3933 


26 


.6600 


45 


•3933 


8 


• 3933 


27 


•6333 


46 


•3933 


9 


.6600 


28 


.6333 


47 


.3667 


10 


.6600 


29 


.6067 


48 


.3667 


11 


.6067 


30 


.6067 


49 


.3400 


IP 


.6067 


31 


.58OO 


50 


.3400 


13 


■ 5533 


32 


.5800 


51 


3133 


14 


•5533 


33 


•5533 


52 


.3133 


13 


.5000 


34 


• 5533 


53 


.7000 


16 


.5000 


35 


.5267 


54 


.7000 


17 


.4467 


36 


.5267 


55 


.6867 


18 


.4467 


37 


.5000 


56 


.6867 


19 


•3933 


38 


.5000 


57 


• 6733 
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DIFFICULTY VALUES FOR TWO- ITEM- PER- STAGE- E, TEN-ITEM BRANCHING TEST 
WITH DIFFICULTY MANGE OF . 30- -70( Continued ) 



:em No. 


Diff. Value 


Item No . 


Diff, Value 


Item No . 


Diff. Value 


58 


• 6733 


77 


.5400 


96 


.4200 


59 


.6600 


78 


.5400 


97 


. 406'(f 


60 


.6600 


79 


.5267 


98 


.4067 


61 


.6*167 


80 


.5267 


99 


•3933 


62 


.6167 


81 


.5133 


100 


• 3933 


63 


• 6333 


82 


■5133 


101 


.3800 


64 


.6333 


83 


.5000 


102 


.3800 


65 


.6200 


84 


.5000 


103 


■ 3667 


66 


.6200 


85 


.4867 


104 


• 3667 


67 


.6067 


86 


.4867 


105 


• 3533 


68 


.6067 


87 


•4733 


106 


• 3533 


69 


•5933 


88 


•4733 


lor 


.3400 


70 


•59 33 


89 


.4600 


108 


.3400 


71 


.5S00 


90 


.4600 


109 


.3267 


72 


.5800 


91 


.4467 


110 


.3267 


73 


• 5667 


92 


.4467 


111 


■ 3133 


74 


.5667 


93 


.4333 


112 


.3133 


75 


• 5533 


94 


•4333 


113 


.3000 


76 


• 5533 


95 


.4200 


114 


.3000 
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DIFFICULTY VALUES FOR TWO-ITEM- 
WITH DIFFICULTY RANGE 


PER- 

OF 


STAGE- U, 
•50- .70 


TEN- ITEM BRANCHING TEST 
(Figure 7 ) 


Item No . 


Dlff. Value 


Item No , 


Dif f . Value Item No . 


Diff. Value 


1 


.5000 


20 




.4500 


39 


.4875 


2 


.5000 


21 




.4000 


40 


.4875 


3 


.5500 


22 




.4000 


41 


.4750 


4 


.5500 


23 




.6500 


42 


• 4750 


5 


.5000 


24 




.6500 


43 


.4625 


6 


.5000 


25 




,6000 


44 


.4625 


7 


. 450 c 


26 




.6000 


45 


.4500 


8 


.4500 


27 




• 5750 


46 


.4500 


9 


.6000 


28 




.5750 


47 


.4250 


10 


.6000 


29 




• 5500 


48 


.4250 


11 


•5500 


30 




•5500 


49 


.4000 


ia 


.5500 


31 




• 5375 


50 


.4000 


13 


.5250 


32 




• 5375 


51 


.3500 


14 


.5250 


33 




.5250 


52 


• 3500 


15 


.5000 


34 




.5250 


53 


.7000 


16 


.5000 


35 




.5125 


54 


.7000 


17 


.4750 


36 




.5125 


55 


.6500 


18 


.47 50 ' 


37 




.5000 


56 


.6500 


19 


.4500 


38 




.5000 


57 


.6250 
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DIFFICULTY VALUES FOR TWO- ITEM- PER-STAGE-U, TEN- ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .30-. 70 (Continued) 



Item No. 


Diff. Value 


Item No. 


Diff. Value 


Item No . 


Diff. Value 


58 


.6250 


77 


.5187 


96 


.4625 


59 


.6000 


7« 


.5187 


97 


.4562 


60 


.6000 


79 


.5125 


98 


.4562 


61 


.5875 


80 


.5125 


99 


.4500 


82 


.5875 


81 


• 5062 


100 


.4500 


63 


•5750 


82 


• 5062 


101 


•4375 


64 


•5750 


83 


.5000 


102 


• 4375 


65 


• 5625 


84 


.5000 


103 


.4250 


66 


•5625 


85 


• 4937 


104 


.4250 


67 


•5500 


86 


.4937 


105 


.4125 


68 


■5500 


87 


.4875 


106 


.4125 


69 


•5437 


88 


.4875 


107 


.4000 


70 


•5437 


89 


.4812 


108 


.4000 


71 


•5375 


90 


.4812 


109 


•3750 


72 


•5375 


91 


.4750 


110 


•3750 


73 


•5312 


92 


• 4750 


111 


•3500 


74 


.5312 


S3 


.4687 


112 


•3500 


75 


•5250 


94 


.4687 


113 


.3000 


76 


.^50 


95 


.4625 


114 


.3000 




- 2C - 



DIFFICULTY VALUES FOR TWO-ITEM- PER-STAGE-E, TEN-ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .10-. 90 (Figure 7 ) 



Item No . 


Diff. Value 


Item No. 


Diff. Value 


Jtem No. 


Diff. Value 


1 


.5000 


20 


.2867 


39 


.4467 


2 


.5000 


21 


.1800 


40 


.4467 


3 


.7133 


22 


.1800 


41 


• 3933 


4 


.7133 


23 


.8733 


42 


•3933 


5 


.5000 


24 


.8733 


43 


.3400 


6 


• 5000 


25 


.8200 


44 


.3400 


7 


.2867 


26 


.(3200 


45 


.2867 


8 


.2867 


27 


•7667 


46 


.2867 


9 


.8200 


28 


.7667 


47 


• 2333 


10 


.8200 


29 


.7133 


48 


• 2333 


11 


.7133 


30 


•7133 


49 


.1800 


12 


.7133 


31 


.6600 


50 


.1800 


13 


.6067 


32 


.6600 


51 


.1267 


14 


.6067 


33 


.6067 


rA 

>- 


.1267 


15 


.5000 


34 


.6067 


53 


.9000 


16 


.5000 


3? 


•5533 


54 


.9000 


17 


• 3933 


36 


• 5533 


55 


•8733 , 


18 


• 3933 


37 


.5000 


56 


• 8733 


19 


.2867 


38 


.5000 


57 


.8467 
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DIFFICULTY VALUES FOR TWO-ITEM- 
WITH DIFFICULTY RANGE 


PER- STAGE- E, TEN- ITEM BRANCHING TEST 
OF .10- .90 (Continued) 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


Item No. 


Diff. Value 


58 


.8467 


77 


.5800 


96 


.3400 


59 


.8200 


78 


.5800 


97 


.3133 


60 


.8200 


79 


.5533 


98 


.3133 


61 


■7933 


80 


•5533 


99 


.2867 


62 


•7933 


81 


.5267 


100 


.2867 


63 


.7667 


82 


.5267 


lol 


.2600 


64 


.7667 


83 


.5000 


102 


.2600 


65 


.7400 


84 


.5000 


103 


• 2333 


66 


. 7400 


85 


• 4733 


104 


• 2333 


67 


.7133 


86 


• 4733 


105 


.206? 


68 


.7133 


87 


.4467 


106 


.2067 


69 


.6867 


88 


.4467 


307 


.1800 


70 


.6867 


89 


.4200 


108 


.1800 


71 


.6600 


90 


.4200 


109 


■1533 


72 


.6600 


91 


•3933 


110 


■1533 


73 


■6333 


92 


•3933 


111 


.1267 


74 


• 6333 


93 


.3667 


112 


.1267 


75 


.'067 


94 


.3667 


113 


.1000 


76 


.6067 


95 


.3400 


114 


.1000 



o 

ERIC 

kr.iiiiiM7ni.TLaitJ 



• 22 



DIFFICULTY VALUES FOR TWO-ITEM- 
WITH DIFFICULTY RANGE 


PER-STAGE-U, 
OF .10- .90 


TEN- ITEM BRANCHING TEST 
(Figure 7 ) 


Item No. 


Diff. Value 


Iteii No. 


Diff. Value Item No. 


Diff. Value 


1 


.5000 


20 


.4000 


39 


.4750 


2 


.5000 


21 


■ .3000 


40 


•4750 


5 


.6000 


22 


.3000 


41 


.4500 


4 


.6000 


23 


.8000 


42 


.4500 


5 


• 5000 


24 


.8000 


43 


.4250 


6 


.5000 


23 


.7000 


44 


.4250 


r? 

1 


.4000 


26 


.7000 


45 


.4000 


8 


.4000 


27 


.6500 


46 


.4000 


9 


.7000 


28 


.6500 


47 


•3500 


10 


.7000 


29 


.6000 


48 


•3500 


11 


.6000 


>3 


.6000 


49 


.3000 


12 


.6000 


31 


•5750 


50 


.3000 


13 


•5500 


32 


•5750 


51 


.2000 


14 


•5500 


33 


•5500 


52 


.2000 


15 


.5000 


34 


•5500 


53 


.9000 


16 


.5000 


35 


.5250 


54 


.9000 


IT 


.4500 


36 


•5250 


55 


.8000 


18 


.4500 


37 


.5000 


56 


.8000 


19 


.4000 


38 


.5000 


57 


•7500 
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DIFFICULTY VALUES FOR TWO-ITEM- PER-STAGE-U , TEN- ITEM BRANCHING TEST 
WITH DIFFICULTY RANGE OF .10- .90 (Continued) 



Item No. 

58 

59 

60 
61 
62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

75 - 5500 

76 .5510 



:em No. 


Diff, Value 


77 


•5375 


78 


■5375 


79 


•5250 


80 


•5250 


81 


.5125 


82 


.5125 


83 


.5000 


84 


.5000 


85 


.4875 


86 


.4875 


87 


.4750 


88 


.4750 


89 


.4625 


90 


.4625 


91 


.4500 


92 


.4500 


03 


• 4375 


94 


• 4375 


95 


.4250 



Item No . 


Diff. Value 


96 


.4250 


97 


.4125 


98 


.4125 


99 


.4000 


loo 


.4000 


101 


•3750 


102 


• 3750 


1 -03 


•3500 


104 


• 3500 


105 


.3250 


106 


.3250 


107 


.3000 


108 


.3000 


109 


.2500 


110 


.2500 


111 


.2000 


112 


.2000 


113 


.1000 


114 


.1000 



Diff. Value 

• 7500 
.7000 
.7000 

• 6750 
•6750 
.65OO 
.6500 
.6250 
.6250 
.6000 
.6000 
.5875 

• 5 375 

• 5750 

• 5750 
■ 5625 

• 562 5 
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COMPUTATIONAL PROCEDURES AND ASSUMPTIONS! 



Statistical computations were based on a theoretic'*! model presented 
by Lord (3)» The model assumes that there is a trait or ability under- 
lying the raw scores on a test, and that the probability of an examinee's 
responding correctly to a test item is a normal ogive function of his 
position on the ability dimension. Since item response ; are a function 
only of scores on the ability continuum, they are independent of each 
other when ability is held constant,, When all the item i in a test are 

2 

assumed to have the same biserial correlation (R^ with ability, R^~ is 

an estimate of item intercorrelation. Three major step 1 ? are involved in 
obtaining the correlation between test score and underlying ability: 

The proportion of examinees passing each idem is determined for each of 
the ability level s under consideration; the conditional distribution of 
test scores is obtained for each ability level; and the bivariate fre- 
quency distribution of test score and ability is obtain'd. 



Proportion of Examinees at a Given Level of Ability Who Pass an Item 

When the group tested is assumed to be normally distributed on 
ability, Lord's formulas ( 9 ) and (10) may be used to find the proportion 
of examinees who pass each of the test items when ability is held con- 
stant. In Lord's notation, a value of (the z score corresponding to 

the p-value of item i at a specified ability level) is computed for each 
ability level under consideration by formula ( 9 ): 

h. - R. • c 
i i , t 

g i , where 



h^ « the z value corresponding to the population p*va ue of item i 
= the biserial correlation between item i and underlying ability 
c = the z score representing the ability level beinj, considered 



= V 1 - 



Each g^ i s converted to (p~value of item i for examinees at a given 
ability level) by Lord’s formula (10): 



1'^ = A (g^) = area of normal curve above the point g^. 
These values are computed for each ability level. 
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Conditional Test Scc-re Distribution for Given Ability Levels 

For conventional tests, the distribution of test scores at each of 
the specified ability levels may be computed by expansion of Lord's 
formula (ll): f 



n 

tt ( P. + Q . ) , where 
i = i 1 1 



n 

tt indicates the successive multiplication of the ( P. + Q ) terms 
i = i ii 

n = number of items in test 

= proportion passing item i for the given ability level 




Terms of this expansion give all possible ways of obtaining various tert 
scores. Those terms which lead to the same test score are summed to ol 
the distribution of test scores for a given ability level. 

Although Lord does not discuss branching tests, his model is also 
applicable to this type of test. For a branching test, the proportioi 
of examinees (at a specified ability level) following any given path r l* 
be determined by multiplying the or values (as obtained by Lord 1 

formulas 9 and 10) of the items which make up that path. If an item if 
passed, its p value is used; if an item is failed, its value is use 

Such a proportion is computed for each path, and values for paths leaU; 
to the same test score are summed to obtain the test score distribute i. 



Bivariate Frequency Distribution of Test Score and Ability 

For both conventional and branching tests, the bivariate distribu^i 
of test score and ability is obtained by multiplying the conditional to. 
score distribution for each ability level by the ordinate value of the 
normal curve at that ability level (Lord's formula 14, applicable when 



0 
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normal distribution of ability is assumed). The test-ability correlation 
coefficient may be computed 3 ^ from this seatterplot. 

In the present study, the distribution of underlying ability in the 
theoretical sample of examinees was assumed to be normal with X = 0 and 
o' - 1*00. Twenty-nine levels of ability, measured in standard scores 
ranging from +3*5 to -3*5 In steps of .25 were used. The biserial corre- 
lation between an item and ability was constant for all items in a given 
test, For each of the five- and ten-i tem-per-sub ject conventional and 
branching tests evaluated, the assumed biserial was varied from #30 to 
.90 in steps of .10. The f i f teen- item tests were evaluated at biserials 
of .40, .60 and .80. 



RESULTS AND DISCUSSION 



Fivedtem Tests 

Conventional Tests . The correlation coefficients between test score 
and ability for the five-item conventional tests are shown in the first 
three rows of Table 2, For biserlals of .30 through .'fO ( r i j “ -09 

through .4Q), the all .50 (C 50 ) test obtained the highest coefficients, 
and the i y- y through .7 CR test yielded a higher relationship to the 
ability criterion than did the ,1 through .9 CR test,* At the .80 bi serial 
(r^ = .64), the .3 through .7 CR test yielded the highest coefficient, 

and the C50 test was next. Finally, at the assumed biserial of .90 
(r^ = . 81 ), the wide range rectilinear test (.1 through #9 CR test) had 

the highest correlation coefficient, and the C 50 test had the lowest 
coefficient of the three conventional tests. Overall, the C50 was best 
for low to moderate Item intercorreiat ion ; the moderate range ( *3 through 
. 7 ) and eventually the wider range ( .1 through . 9 ) tests were best for 
higher intercorrelations. 



^ A FORTRAN program which performs these computations was written for the 
GE 22 5 computer^ by Mr. Sidney Sachs of the Computer Applications Branch 
Behavior and Systems Research Laboratory. This program was used 
to obtain the test-ability coefficients reported. It should be noted 
that Brogden (4), Tucker ( 5 ), and Lord ( 3 ) have provided computationally 
easier formulas for obtaining the test-ability coefficients for conven- 
tional tests, 

^The commercial designati ... n Is used only in the interest of specificity 
In reporting. its use does not constitute indorsement by the Army or 
by BESRL. 

^ For simplicity in presentation, difficulty values mentioned in the text 
are hereafter expressed in one decimal place In contrast to the biserial 
which are in two decimal places. 



Branching Tests . The results for the f ive-item-per-subject branch- 
ing tests are shown in the last two rows of Table 2 . The coefficient for 
the moderate range .3 through ' .7 test was higher than that for the wider 
range test for assumed biserials of .30 through .80; the .1 through .9 
range test had the higher coefficient at r^ ig = ,90. 

Comparison of Conventional and Branching Test s. One of the branch- 
ing tests was superior to any of the conventional tests for r^ £ .60 

( r ^ s . 36 ). At the higher biserials, .70 through .90, both branching 

tests yielded higher coefficients than did any of the conventional tests. 
For the lower biserials, .30 through .50, the C 50 conventional tests re- 
sulted in slightly higher coefficients than did either of the branching 
tests . 



Table 2 



TEST SCORE-ABILITY CORRELATION COEFFICIENTS FOR FIVE-ITEM- PER-SUBJECT 
CONVENTIONAL AND BRANCHING TESTS 



Biserial s 


.30 


.40 


• 50 


.60 


.70 


.80 


.90 


Te. t 




Correlation Coef f i c ient s* 






C (all .50) 


482 


601 


696 


769 


823 


858 


671 


c ( -3 - .7, R) 


473 


591 


686 


762 


819 


861 


887 


C ( .1 - .9, R) 


434 


549 


646 


726 


793 


850 


900 


B (-3 * - 7 , 1 * PS) 


478 


599 


694 


774 


835 


880 


906 


B ( .1 - .9, 1 -PS) 


461 


580 


60O 


760 


826 


878 


Q20 



a Oec»maf Pcinls omitted. 



Ten-Item Tests 

Conventional Tests . The test score- abi l i ty correlation coefficients 
for the ten-item conventional tests are shown in the first five rows of 
Table 3 • The C 50 test had the highest coefficient for each biserial 
through . 60 . For these same biserials, all the .3 through . f range tests 
were next highest and the .1 through .9 range tests were lowest. At 
r^ * .70, the C50 and .30 through .7 tests were about equally effective, 

and yielded higher coefficients than the .1 through .9 tests. At biserials 
.80 and . 90 , the original situation was reversed and the C50 test had the 
lowest coefficients and the .1 through .9 tests the highest cooff ic i enr s . 
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Branca ing Tests . The ten-item-per-sub j ect branching test data are 
given in the bottom six rows of Table J- The .3 through .7 1 -PS tests 
tended to correlate higher with the criterion than did the *1 through .9 
tests through a biserial of . 60 * Above this level the converse held. It 
should be noted that for all biserials, and any given item difficulty 
range, the 1 -PS branching test correlated higher chan any 2 -PS test 
covering the same range. In fact, with only one exception (r^ g = * 90 ), 

both the 1 -PS .1 through «9 and «3 through .7 tests yielded higher 
coefficient; than did any of the 2 -PS tests. The 2 -PS-E tests correlated 
higher with the ability criterion than did the 2 -PS-U tests. 



Table 3 

TEST SCORE-ABILITY CORRELATION COEFFICIENTS FOR TEN- ITEM- PER-SUBJECT 
CONVENTIONAL AND BRANCHING TESTS 



Biserial s 


.50 


.40 


.50 


.60 


•70 


.80 


.90 


Test 






Correlation Coefficients 4 




C (all .5) 


614 


728 


807 


850 


891 


905 


898 


C ( .5 - . 7 , N) 


608 


723 


802 


856 


890 


909 


910 


C ( .5 - - 7 , R) 


604 


719 


799 


854 


890 


911 


917 


C ( .1 - . 9 , N) 


586 


702 


786 


844 


886 


915 


929 


C ( .1 - .9. R) 


583 


680 


767 


830 


877 


915 


941 


B ( - . 7 , i-Ps) 
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728 
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866 
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951 


B ( .5 - - 7 , 2 -PS-E) 
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642 


757 


809 


863 


898 


915 


B { .5 - . 7 . 2-PS-U) 


512 


635 


721 


799 


851 


685 


898 


B ( .1 - .9, 1 -PS) 


601 


719 


801 


862 


905 


954 


955 


B ( .1 - . 9 , 2 -PS-E) 


551 


655 


751 


825 


881 


921 


948 


B ( .1 - .i, 2 -PS-U) 


519 


640 


729 


808 


862 


899 


918 


a Decimal points omitted. 
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at r, , = .40 the CbO conventional and .3 through 

bis 

had the largest coefficients. The 2-PS branching 
with the best conventional test only at very high 



.7 1- PS branching tests 

tests compared favorably 
bi serial s . 



Fifteen -I tern Tests 

Conventional Tests . All fifteen-item tests were evaluated at biserials 
of .40, .60, and .80- The test s^ore-abil i ty correlation coefficient for 
the five conventional tests are given in the first five rows of Table 4. 

The C 50 test had the highest coefficient at biserials of .40 and .60. At 
a biserial of .80, the .1 through .9 tests (both N and R) did be ?t . A 
comparison of the .3 through .7 and .1 through .9 tests across tl e three 
biserials showed that the narrower range tests received higher coeffi- 
cients at the lower biserials ( .40 and .60) and the wider range tests did 
better for the high biserial (.80). This general trend was consistent 
with the results obtained for the five- and ten-item conventional tests. 

For tests of a given range of item difficulties, those with approximately 
normally distributed item difficulties were superior to those with recti- 
linear difficulty distributions at the .40 biserial and did less well than 
their rectilinear counterparts at a biserial of ,80. At r^ = *60, no 

difference was obtained between the .3 through .7 N and R tests, but the 
.1 through .9 N test was superior to the R test of the same range. This 
same trend was also found for the ten-item convantional tests. In 
general, as biserials (and thus item intercorrelations) increased, wider 
range tests and test 3 with more rectilinear item difficulty distributions 
did progressively better. 

Branching Tests . Data for the two fifteen-item branching tests are 
given in the last two rows of Table 4. The .3 through .7 test correlated 
higher with the ability criterion at the .40 biserial, while the .1 
through .9 test yielded the highest coefficient at r^ s ~ *80* The two 

branching tests were essentially equivalent at the .60 biserial. 



Comparison of Conventional and Branching Tests . Both branching tests 
yielded higher coefficients than did any of the conventional tests for 
biserials of .60 and , 80 . At the .40 biserial, the C 50 test was essentially 
equivalent to the .3 through .7 branching test. 



Effects of Test Length 

Table 5 gives the increments in test score-ability coefficients as 
the tests were increased in length from five to fifteen items. Increasing 
the number of items from five to ten resulted in increments in correlation 
about twice as large as those obtained by increasing test length from ten 
to fifteen items. Increases in test length led to higher test score-ability 
coefficients for the lower biserial values. There appeared to be little 
difference between conventional and branching tests in terms of the effects 
of increasing test length. 



TabI e 4 




TEST SCORE-ABILITY CORRELATION COEFFICIENTS FOR FIFTEEN- ITEM- PER- SUBJECT 
CONVENTIONAL AND BRANCHING TESTS 
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OVtRVIE V 



Both conventional and branching test data showed that tests with the 
least spread of item difficulties yielded the highest correlation coeffi- 
cients with underlying ability when low to moderate item biserials were 
assumed. For medium to high biserials, the moderate range and wide range 
tests tended to yield coefficients of about the same magnitude. The wide 
range tests generally did best when very high biserials were assumed. 

These data are consistent with the 9“ and l 8 -item test data reported by 
Brogden (4). The shift in the relative effectiveness of the narrower and 
wider range tests tended to take place earlier when test length was 
increased • 

For the lowest biserial assumed (.30), the C 50 test was the only con- 
ventional test which correlated higher with the ability criterion than did 
the best branching test. At biserials of .40 and . 50 , the ten- and 
fifteen-item branching tests covering a .3 to .7 range and the C 50 test 
were essentially equivalent. For biserials of ,60 and above, one of the 
branching tests always did better than any of the conventional tests. In 
general, the differences in correlation with underlying ability were small 
but systematic. Since the data were by definition errorless, greater 
significance may be attached to these differences than would be the case 
with empirical data, 

A comparison of one- item-per- stage and two- item-per- stage branching 
tests (at the ten-item test length) indicated that the one- i tern- per-stage 
tests had uniformly higher coefficients than did the two-item-per-stage 
tests of the same range. In view of these results, it would not seem 
profitable to use the more complex two-item-per-stage structure in the 
development of branching tests for the purpose of maximizing overall 
correl at ion . 
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